1. Field of the Invention
The present invention relates to an image processing device, computer program product and an image processing method for generating motion vectors based upon image data, for example.
2. Discussion of the Background
Conventionally, block matching has been known as a method for detecting motion vectors between frames of an image. FIG. 1 is a diagram that illustrates a general image processing device, and FIG. 2 is a flowchart that illustrates the operation of the general image processing device shown in FIG. 1.
In reference to FIGS. 1 and 2, “Block matching” is a process wherein a motion vector is obtained, which indicates what position a certain motion block, in an image at a certain point in time T−1, moves to at a next point in time T.
For example, an image processing device 1s (FIG. 1) comprises frame memory 3s-T, frame memory 3s-T−1, search region memory ms, reference block memory mr, phase block memory mp, computation unit pc, evaluation table memory mv, and minimal value search unit ps.
The image processing device 1s sets a search region at a point in time T so as to perform matching for pixels in a block within the search region with regard to each phase (position) as to a reference block at a point in time T−1, and the phase scoring of the best match is taken as a position of the block at the point in time T. Here, the shift amount of the block between two frames is defined as the motion vector.
Referring to FIG. 2, a flowchart describes the operation of the image processing device 1s. The image data of the search region and the image data of reference block br are each read out from frame memory T and frame memory T−1 wherein continuous input images are stored with regard to time (ST1, ST2). These image data sets are stored in the search region memory ms and the reference block memory mr (ST3).
Next, the block with the same shape and the same size as the reference block br is read out from the search region memory ms for each phase. The block is defined as a phase block bp. The phase is sequentially updated at a point in time when recording to the evaluation table has ended, as described later, and the phase block bp is read out at the phase at the point of time (ST4). The phase block bp read out is stored in the phase block memory mp as shown in FIG. 1, for example.
FIG. 3 (which has two parts—FIG. 3a and FIG. 3b) is a diagram for describing a specific process example beginning with how a readout of the phase block operation is performed with phase shift, up to recording in the evaluation table memory mr. FIG. 3(a) is a diagram which illustrates the state prior to the phase shift, and FIG. 3(b) illustrates the state following the phase shift.
Referring to FIG. 3, the operations up to, and including, recording in the evaluation table memory will be described. Presume that the size of the reference block is 3×3 pixels, and the size of the search region is 5×5 pixels. Here, the phase block bp read out at the point of the phase 1 is shifted in phase 2 to a position offset by one pixel in the horizontal direction. For example, the shift amount of the phase due to updating is one pixel. Next, the absolute value of the difference is computed by the computation unit pc for each pair of corresponding pixels between the reference block br and phase block bp, the sum of absolute value of the difference S is obtained for all the absolute values of difference computed.
                    S        =                              ∑                          s              ,                              p                =                1                                      9                    ⁢                                          ⁢                                                                k                s                            -                              k                p                                                                                    (        1        )            
The sum of absolute value of the difference S is calculated with Expression (1), for example, using the pixel values ks in the reference block br and the corresponding pixel values kp in the phase block bp (ST5).
Next, the sum of absolute value of the difference S is stored in the evaluation table (ST6). Specifically, the sum of absolute values of difference S is stored in the evaluation table memory mv, as shown in FIG. 1. The evaluation table stores the sum of absolute values of difference S for each phase corresponding to the phase block read out.
In general, with the image processing device 1s, upon the phase being shifted from the shift 1 to the shift 2, the position at which the data is stored in the evaluation table is shifted corresponding to the shift amount as shown in FIG. 3.
Upon the processing from readout of phase blocks bp up to recording in the evaluation table ending for all the search regions (ST7), the minimal value search unit ps searches the minimal value within the evaluation table (ST8).
On the other hand, in the event that judgment is made that the evaluation table does not store the data for all the phases in Step ST7, the flow returns to processing in Step S3.
Thus, the image processing device 1s searches the block within the search region, of which value is closest to that of the reference block br.
As described above, the evaluation table stores the sum-of-absolute-values-of-difference S for each phase corresponding to the phase block bp read out. Accordingly, the phase of the block within the search region, corresponding to the reference block br, can be found by searching the minimal value within the evaluation table. The information with regard to the phase is output as a motion vector.